{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stress Test性能测试\n",
    "- 千帆 Python SDK 提供了基于locust工具的对大模型服务进行快速压测以及性能评估的功能。该功能入口在Dataset对象的stress_test方法中。\n",
    "\n",
    "- 本文提供了一个使用stress_test进行压测的示例，通过本文，您可以快速上手我们的压测工具。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 由于Locust中使用了gevent库来保证高并发性能，而gevent的高并发依赖于monkey patching的非阻塞I/O机制，但该机制在Notebook环境中默认未开启。因此，在开始测试前，需要进行monkey patching操作。这样做是为了确保整个环境，包括IPython/Jupyter自己的组件，都使用gevent兼容的非阻塞版本，从而避免因混合使用阻塞和非阻塞操作导致的不一致性和潜在的死锁问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gevent import monkey\n",
    "monkey.patch_all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始压测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING][2024-06-27 15:19:50.157] chunk_reader.py:36 [t:4407381824]: ijson aren't installed, only online function can be used\n",
      "[INFO][2024-06-27 15:19:50.209] dataset.py:408 [t:4407381824]: no data source was provided, construct\n",
      "[INFO][2024-06-27 15:19:50.222] dataset.py:276 [t:4407381824]: construct a file data source from path: /Users/jianruitian/Downloads/bce-qianfan-sdk-main/cookbook/stress/st.jsonl, with args: {}\n",
      "[INFO][2024-06-27 15:19:50.238] file.py:293 [t:4407381824]: use format type FormatType.Jsonl\n",
      "[INFO][2024-06-27 15:19:50.267] utils.py:349 [t:4407381824]: start to get memory_map from /Users/jianruitian/.qianfan_cache/dataset/Users/jianruitian/Downloads/bce-qianfan-sdk-main/cookbook/stress/st.arrow\n",
      "[INFO][2024-06-27 15:19:50.289] utils.py:277 [t:4407381824]: has got a memory-mapped table\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         0     0(0.00%) |      0       0       0      0 |    0.00        0.00\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |   3216    3216    3216   3216 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |   3216    3216    3216   3216 |    0.00        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |   1032    1032    1032   1032 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |   1032    1032    1032   1032 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |     72      72      72     72 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |     72      72      72     72 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |     33      33      33     33 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |     33      33      33     33 |    0.00        0.00\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |   3216    3216    3216   3216 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |   3216    3216    3216   3216 |    0.00        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |   1032    1032    1032   1032 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |   1032    1032    1032   1032 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |     72      72      72     72 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |     72      72      72     72 |    0.00        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          1     0(0.00%) |     33      33      33     33 |    0.00        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         1     0(0.00%) |     33      33      33     33 |    0.00        0.00\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |   3345    3216    3474   3216 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |   3345    3216    3474   3216 |    0.20        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |   1068    1032    1104   1032 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |   1068    1032    1104   1032 |    0.20        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |     72      72      72     72 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |     72      72      72     72 |    0.20        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |     33      33      34     33 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |     33      33      34     33 |    0.20        0.00\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |   3345    3216    3474   3216 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |   3345    3216    3474   3216 |    0.20        0.00\n",
      "\n",
      "\u001b[0;36m首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |   1068    1032    1104   1032 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |   1068    1032    1104   1032 |    0.20        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |     72      72      72     72 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |     72      72      72     72 |    0.20        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36m输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          2     0(0.00%) |     33      33      34     33 |    0.20        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         2     0(0.00%) |     33      33      34     33 |    0.20        0.00\n",
      "\u001b[0m\n",
      "[2024-06-27 15:20:00,948] \u001b[0;33m[NOTICE] Distributor[distributor] StopIteration occurs 1 times, and 1 users have been spawned.\u001b[0m\n",
      "\u001b[0;36mSummary: 首token延迟时间统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          3     0(0.00%) |   1048    1007    1104   1007 |    0.29        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         3     0(0.00%) |   1048    1007    1104   1007 |    0.29        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36mSummary: 首token延迟时间统计 [percentiles (approximated)]\n",
      "Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "POST     ERNIE-Bot                                                                            1000   1000   1100   1100   1100   1100   1100   1100   1100   1100   1100      3\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "         Aggregated                                                                           1000   1000   1100   1100   1100   1100   1100   1100   1100   1100   1100      3\n",
      "\u001b[0m\n",
      "\u001b[0;36mSummary: 输入token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          3     0(0.00%) |     72      72      72     72 |    0.29        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         3     0(0.00%) |     72      72      72     72 |    0.29        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36mSummary: 输入token数统计 [percentiles (approximated)]\n",
      "Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "POST     ERNIE-Bot                                                                              72     72     72     72     72     72     72     72     72     72     72      3\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "         Aggregated                                                                             72     72     72     72     72     72     72     72     72     72     72      3\n",
      "\u001b[0m\n",
      "\u001b[0;36mSummary: 输出token数统计 [stats]\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          3     0(0.00%) |     33      32      34     33 |    0.29        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         3     0(0.00%) |     33      32      34     33 |    0.29        0.00\n",
      "\u001b[0m\n",
      "\u001b[0;36mSummary: 输出token数统计 [percentiles (approximated)]\n",
      "Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "POST     ERNIE-Bot                                                                              33     33     34     34     34     34     34     34     34     34     34      3\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "         Aggregated                                                                             33     33     34     34     34     34     34     34     34     34     34      3\n",
      "\u001b[0m\n",
      "Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "POST     ERNIE-Bot                                                                          3     0(0.00%) |   3387    3216    3474   3474 |    0.29        0.00\n",
      "--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------\n",
      "         Aggregated                                                                         3     0(0.00%) |   3387    3216    3474   3474 |    0.29        0.00\n",
      "\n",
      "Response time percentiles (approximated)\n",
      "Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "POST     ERNIE-Bot                                                                            3500   3500   3500   3500   3500   3500   3500   3500   3500   3500   3500      3\n",
      "--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------\n",
      "         Aggregated                                                                           3500   3500   3500   3500   3500   3500   3500   3500   3500   3500   3500      3\n",
      "\n",
      "[2024-06-27 15:20:01,094] \u001b[0;32m[INFO] run completed.\u001b[0m\n",
      "[2024-06-27 15:20:01,112] \u001b[0;32m[INFO] Log path: record/20240627T151950Z/run.log\u001b[0m\n",
      "[2024-06-27 15:20:01,123] \u001b[0;32m[INFO] Load Test Statistics\n",
      "QPS: 0.29\n",
      "RPM: 11.29\n",
      "Latency Avg: 3.35\n",
      "Latency Min: 3.22\n",
      "Latency Max: 3.47\n",
      "Latency 50%: 3.22\n",
      "Latency 80%: 3.5\n",
      "FirstTokenLatency Avg: 1.07\n",
      "FirstTokenLatency Min: 1.03\n",
      "FirstTokenLatency Max: 1.1\n",
      "FirstTokenLatency 50%: 1.03\n",
      "FirstTokenLatency 80%: 1.1\n",
      "InputTokens Avg: 72.0\n",
      "OutputTokens Avg: 33.5\n",
      "TotalQuery: 3\n",
      "SuccessQuery: 2\n",
      "FailureQuery: 1\n",
      "TotalTime: 10.62\n",
      "SuccessRate: 66.67%\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "# 开启压测服务\n",
    "os.environ['QIANFAN_ENABLE_STRESS_TEST'] = \"true\"\n",
    "\n",
    "from qianfan.dataset import Dataset\n",
    "\n",
    "# 请在此处填写您的ak、sk\n",
    "os.environ[\"QIANFAN_ACCESS_KEY\"] = \"...\"\n",
    "os.environ[\"QIANFAN_SECRET_KEY\"] = \"...\"\n",
    "\n",
    "# 需要初始化一个数据集对象\n",
    "ds = Dataset.load(data_file=\"/data_file/stress_test.jsonl\")\n",
    "\n",
    "ds.stress_test(\n",
    "    users=1,\n",
    "    workers=1,\n",
    "    spawn_rate=128,\n",
    "    model=\"ERNIE-Bot\",\n",
    "    model_type=\"ChatCompletion\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下是各参数的说明：\n",
    "\n",
    "- **workers (int)**：指定发压使用的worker数目，每个worker为1个进程，默认为1个进程；\n",
    "- **users (int)**：指定发压使用的总user数，必须大于worker数目；每个worker负责模拟${users}/${workers}个虚拟用户；\n",
    "- **runtime (str)**：指定发压任务的最大运行时间，格式为带时间单位的字符串，例如（300s, 20m, 3h, 1h30m）；压测任务启动后会一直运行到数据集内所有数据都请求完毕，或到达该参数指定的最大运行时间；该参数默认值为'0s'，表示不设最大运行时间；\n",
    "- **spawn_rate (int)**：指定每秒启动的user数目；\n",
    "- **model (str)**：指定需要压测服务的模型名称。该参数与endpoint只能指定一个；\n",
    "- **endpoint (str)**：指定需要压测服务的url路径。该参数与model只能指定一个；\n",
    "- **model_type (str)**：指定被测服务的模型类型。 目前只支持'ChatCompletion'与'Completion两类'；默认值为'ChatCompletion'；\n",
    "- **hyperparameters (Optional[Dict[str, Any]])**：指定压测时使用的超参数；"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各项指标含义如下：\n",
    "\n",
    "- **QPS**：服务每秒实际处理的请求数；\n",
    "- **RPM**：每分钟实际处理的请求数；\n",
    "- **Latency Avg/Min/Max/50%/80%**：全长时延的平均值/最小值/最大值/50分位值/80分位值；\n",
    "- **FirstTokenLatency Avg/Min/Max/50%/80%**：首句时延的平均值/最小值/最大值/50分位值/80分位值；\n",
    "- **InputTokens Avg**：单次请求输入的token长度平均值；\n",
    "- **OutputTokens Avg**：单次请求输出的token长度平均值；\n",
    "- **TotalQuery/SuccessQuery/FailureQuery**：总请求数/成功请求数/失败请求数；\n",
    "- **TotalTime**：总运行时间；\n",
    "- **SuccessRate**：请求成功率；"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您也可以在当前路径下的record中查看相应的log文件。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
